Adaptive image processing unit for extracting characteristic portion from image

ABSTRACT

An image area extract unit includes a modeling unit for generating a pixel evaluation model used for extraction, and an extract unit for extracting a characteristic portion from an original image using the pixel evaluation model. The modeling unit sequentially generates a plurality of partial polynomials according to a modified Group Method of Data Handling using a training image taken under the same condition under which the original image is taken and a supervisory image that designates the characteristic portion of the training image. Each of the generated partial polynomials is outputted to the extract unit in the form of its coefficients only when a square error satisfies a predetermined criterion. The extract unit calculates the feature values for each pixel of the original image using the pixel evaluation model, and defines the extractive area that includes the characteristic portion based on the feature values.

CROSS REFERENCE TO RELATED APPLICATION

[0001] The present application is based on and incorporates herein byreference Japanese Patent Application No. 2001-58497 filed on Mar. 2,2001.

BACKGROUND OF THE INVENTION

[0002] 1. Field of the Invention

[0003] The present invention relates to an image recognition technique,more particularly to an image area extract unit for extracting acharacteristic portion from an image.

[0004] 2. Related Art

[0005] It is proposed that a characteristic portion is extracted from animage of a vehicle, a traffic sign, an advertising display, a person orthe like utilizing image recognition. JP-A-2000-22929 proposes atechnique for extracting the face area from an image of a person in aperson image processing unit. This technique extracts the face area bydetermining whether the RGB values of each pixel satisfy predeterminedfixed criteria for flesh color.

[0006] However, the color of the face area in the image (i.e., the RGBvalues of pixels belonging to the face area) may vary depending on acondition under which the image is taken. Therefore it is difficult toset the fixed criteria for flesh color appropriately. That is, it isevery possibility that the technique cannot extract the face areaproperly depending on a condition under which the image is taken.

[0007] Accordingly, in order to extract the face area properly, thecriteria for flesh color should be adjusted according to a conditionunder which the image is taken. However, it is practically impossible toset different criteria for all possible conditions.

SUMMARY OF THE INVENTION

[0008] The present invention has an object to provide an image areaextract unit capable of adaptively changing a model used for extractionaccording to a condition under which an image is taken so that acharacteristic portion of the image is properly extracted.

[0009] An image area extract unit according to the present inventionincludes original image acquisition means, training image acquisitionmeans, supervisory image acquisition means, model generation means, andarea definition means. The original image acquisition means acquires anoriginal image from which a characteristic portion should be extracted.The training image acquisition means acquires a training image takenunder the same condition under which the original image is taken. Thesupervisory image acquisition means acquires a supervisory image thatdesignates a characteristic portion of the training image. Thecharacteristic portion of the training image corresponds to thecharacteristic portion of the original image. Each pixel of thesupervisory image provides a supervisory output.

[0010] The model generation means generates a pixel evaluation modelbased on the relationship between the values of the pixels of thetraining image and the supervisory outputs provided by the pixels of thesupervisory image. The pixel evaluation model receives the value of oneof the pixels of the original image as an input, and outputs a featurevalue of the pixel. The area definition means calculates the featurevalue of each pixel of the original image using the pixel evaluationmodel, and defines an extractive area of the original image based on thefeature value of each pixel of the original image. The extractive areaincludes the characteristic portion of the original image.

[0011] Preferably, the model generation means sequentially generates aplurality of partial polynomials according to a Group Method of DataHandling. Thus an estimation model that includes at least one of theplurality of partial polynomials is generated as the pixel evaluationmodel.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other objects, features and advantages of thepresent invention will become more apparent from the following detaileddescription made with reference to the accompanying drawings. In thedrawings:

[0013]FIG. 1 is a functional block diagram showing an image processingunit according to an embodiment of the present invention;

[0014]FIG. 2A is a pictorial diagram showing an example of an originalimage inputted to the image processing unit;

[0015]FIG. 2B is a pictorial diagram showing an example of an imageextracted from the original image;

[0016]FIGS. 3A and 3B are pictorial diagrams showing examples of asupervisory image and the corresponding training image used for modelgeneration;

[0017]FIG. 4 is a flowchart of a model generation process executed by amodeling unit of the image processing unit;

[0018]FIG. 5 is a schematic diagram showing how the model generationprocess generates a pixel evaluation model;

[0019]FIG. 6 is a flowchart of an image area extract process executed byan extract unit of the image processing unit;

[0020]FIGS. 7A and 7B are schematic diagrams showing how partialpolynomials are generated during the model generation according to amodified GMDH and a GMDH, respectively;

[0021]FIGS. 8A and 8B are graphs showing relationships between thenumber of partial polynomials generated during the model generation andthe precision of the model according to the modified GMDH and the GMDH,respectively;

[0022]FIG. 9 is a table showing the number of the partial polynomials,the number of additions and multiplications required for achieving eachof five discrete square errors according to the GMDH and the modifiedGMDH;

[0023]FIG. 10 is a schematic diagram showing a neural network employedfor implementing the pixel evaluation model according to a modificationof the embodiment; and

[0024]FIG. 11 is a schematic diagram showing how the GMDH generallygenerates an estimation model.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0025] Referring to FIG. 1, an image processing unit 1 according to anembodiment of the present invention includes a modeling unit 10 and anextract unit 20. The modeling unit 10 includes a supervisory imageacquisition block 11, a training image acquisition block 12 and a modelgeneration block 13. The extract unit 20 includes an original imageacquisition block 21, a feature value calculation block 22, an extractinstruction block 23 and an image output block 24. A memory unit 30,first and second digital video cameras 40, 50 and a monitor 60 areconnected to the image processing unit 1.

[0026] In the extract unit 20, the original image acquisition block 21acquires, as an original image, a color image of a person shown in FIG.2A by the second video camera 50. The data of the original imageincludes an 8-bit Red (R) value, an 8-bit Green (G) value and an 8-bitBlue (B) value for each pixel. The feature value calculation block 22and the extract instruction block 23 together define, as an extractivearea, a rectangular area including the person's face area in theoriginal image. The image output block 24 extracts an imagecorresponding to the defined rectangular area from the original image,and outputs it to the monitor 60. Thus the extract unit 20 outputs animage in which portions other than the extracted area are masked asshown in FIG. 2B.

[0027] Specifically, the feature value calculation block 22 calculates afeature value for each pixel of the original image using a pixelevaluation model and the RGB values of the pixel. The extractinstruction block 23 generates, based on the feature values, extractinstruction data that designates the rectangular area to be extracted.The image output block 24 extracts and outputs the image of thedesignated rectangular area based on the extract instruction data.

[0028] The pixel evaluation model is generated by the modeling unit 10.The supervisory image acquisition block 11 acquires a supervisory imageshown in FIG. 3A from memory unit 30, while the training imageacquisition block 12 acquires a training image shown in FIG. 3B by thefirst video camera 40. The model generation block 13 generates the pixelevaluation model using the supervisory image and the training image.

[0029] The training image is a color image of a person taken under thesame condition under which the original image is taken. The trainingimage should include an image of a person as shown in FIG. 3B. However,it may include an image of a doll instead of that of the person. In thiscase, the RGB values of pixels belonging to the face of the doll shouldbe close to RGB values of pixels belonging to the face of the person.

[0030] On the other hand, the supervisory image is a monochrome image inwhich a portion corresponding to a characteristic portion (i.e., theface area) of the training image is white. However, the white portion ofthe supervisory image is not required to exactly correspond to the facearea of the training image. The supervisory image includes the samenumber of pixels as the training image. The data of supervisory imageincludes an 8-bit value for each pixel. Specifically, in the supervisoryimage, the 8-bit value of each pixel belonging to the portioncorresponding to the characteristic portion of the training image is setto “255”, while the 8-bit value of each pixel belonging to the otherportion is set to “0” as shown in FIG. 3A.

[0031] The model generation block 13 generates the pixel evaluationmodel according to a modification of Group Method of Data Handling(GMDH). The GMDH, which is modeled after a method for breeding seeds, isknown as a method for modeling the relationship between input and outputvariables in a nonlinear system into a polynomial expression (See“Memoirs of Conference on Fuzzy Theory in Japan (vol. 17, No. 2, pp.270-274, 1995)” or “system and control (vol. 23, No. 12, pp. 710-717,1979)”).

[0032] The GMDH generates layered quadratic polynomials each of whichincludes two variables on the right side as follows. Referring to FIG.11, partial polynomials corresponding to the respective pairs of inputvariables x₁ -x_(n) are first generated. That is, _(n)C₂ polynomials,where “n” represents the number of input variables x₁ -x_(n), aregenerated as first-layer partial polynomials. The generated partialpolynomials are trained using training data. Thereafter, at step A, thetrained partial polynomials are evaluated using test data. Further it isdetermined at step A whether a predetermined criterion for terminationof the model generation process is satisfied. If yes, the most superiorpartial polynomial is selected from the generated partial polynomials asan ultimate estimation model and thereafter the model generation processterminates. If not, m₁ partial polynomials (m₁ is a predetermined fixednumber) are selected from the _(n)C₂ partial polynomials based on theresult of the evaluation. The rest of the _(n)C₂ polynomials arediscarded.

[0033] Next _(m) ₁ C₂ polynomials are generated as second-layer partialpolynomials. Each of the generated partial polynomials includes outputvariables of two first-layer polynomials as input variables (i.e., onthe right side). The generated partial polynomials are trained using thetraining data. Thereafter, at step B, the trained partial polynomialsare evaluated using the test data, and it is determined whether thepredetermined criterion for termination of the model generation processis satisfied. If yes, the most superior partial polynomial is selectedfrom m₁ first-layer partial polynomials as the ultimate estimation modeland thereafter the model generation process terminates. If not, m₂partial polynomials (m₂ is a predetermined fixed number)are selectedfrom the _(m) ₁ C₂ polynomials based on the result of the evaluation.The rest of the _(m) ₁ C₂ polynomials are discarded.

[0034] Thus the process is repeated for each layer. When thepredetermined criterion for termination of the model generation processis satisfied, the most superior partial polynomial is selected from thepartial polynomials of the previous layer as the ultimate estimationmodel. That is, the output of the selected partial polynomial isprovided as the output of the ultimate estimation model. The partialpolynomials of further previous layers involved in the selected partialpolynomial are also selected as the partial polynomials included in theultimate estimation model. Then the model generation process terminates.Thus the GMDH generates the estimation model for a non-linear system inthe form of layered partial polynomials. According to the GMDH,computation is relatively complex especially when the number of theinput variables is large.

[0035] Accordingly, the model generation block 13 generates the pixelevaluation model according to the modified GMDH, which generates thereduced number of polynomials. The modified GMDH generates only onepartial polynomial for each layer, and the generated partial polynomialis employed only when the square error is reduced. The modified GMDH iseffective especially in the case that the number of input variables islarge (e.g., 13).

[0036] The model generation block 13 generates the pixel evaluationmodel based on the pixel-by-pixel relation between data of the trainingimage and that of the supervisory image as follows in, for example, tenminutes. First it is assumed that the relationship f between the threeinput variables x₁, x₂, x₃ and the output y can be expressed as:

y=f(x ₁ , x ₂ , x ₃)  (1)

[0037] where the input variables x₁, x₂, x₃ correspond to the RGB valuesof a pixel of an image and the output y corresponds to the feature valuethat indicates whether the pixel belongs to the characteristic portion.

[0038] When input/output data corresponding to N pixels are given, someof the given data is used for training the model expressed as formula(1) and some of the given data is used for evaluating the trained model.For example, data corresponding to Nt pixels are used for training,while data corresponding to N_(e)(=N−N_(t)) pixels are used forevaluation. The given data may be divided into training data and testdata (i.e., data for evaluation) regularly or randomly, or based on thevariance of the given data. Further all given data corresponding to Npixels may be used for both training and evaluation.

[0039] In the present embodiment, the training image data of N pixelsare given as input data, while the supervisory image data of N pixelsare given as output data. Therefore these given data are appropriatelydivided into training data and test data as described above. That is,some pixels of the training image and supervisory image are used as thetraining data and some pixels of the training image and supervisoryimage are used as the test data.

[0040] Referring to FIGS. 4 and 5, three different variables x_(p),x_(q), x_(r) are randomly selected as input variables from an inputvariable group at step 100. When step 100 is first performed, the inputvariable group includes only three variables x₁, x₂, x₃ corresponding tothe RGB values of the training image. Therefore the three variables x₁,x₂, x₃ corresponding to the RGB values are selected as the inputvariables x_(p), x_(q), x_(r). Next a partial polynomial is generatedusing the input variables x_(p), x_(q), x_(r) at step 110. The generatedpartial polynomial is expressed as:

z=c ₀ +c ₁ x _(p) +c ₂ x _(q) +c ₃ x _(r) +c ₄ x _(p) ² +c ₅ x _(q) ² +c₆ x _(r) ² +c ₇ x _(p) x _(q) +c ₈ x _(p) x _(r) +c ₉ x _(q) x _(r)  (2)

[0041] where c₀, c₁, . . . , c₉ are coefficients. The variable z isreferred to as an intermediate variable.

[0042] The coefficients c₀, c₁ . . . , c₉ are determined using linearregression analysis so that a square error E is minimized. The squareerror E is expressed as:

E=Σ(y[i]−z[i])²  (3)

[0043] where z[i] is a value of z when the RGB values of the i-th pixelof the training image is substituted in formula (2), y[i] is the 8-bitvalue of the i-th pixel of the supervisory image, and the symbol “Σ”represents the summation for i =1, 2, . . . , N_(t). The variable irepresents each of the pixels which are selected as training data fromthe training image or the supervisory image. The 8-bit value of the i-thpixel of the supervisory image is thus used as a desired output of thepixel evaluation model when the RGB values of the i-th pixel of thetraining image are substituted in formula (2).

[0044] At step 120, the square error E_(u). subject to evaluation, thatis, the square error given by formula (3) after the coefficients c₀, c₁,. . . , c₉ are fixed, is calculated using the test data. In this casethe variable i represents each of the pixels which are selected as thetest data from the training image or the supervisory image. At step 130,the square error E_(u), is compared with a current least square errorE_(min) which is the minimum of the square errors E_(u) of the partialpolynomials which have been already generated. It is determined whetherE_(u)<E_(min) is satisfied at step 140. If yes (i.e., it is determinedat step 140 that E_(u)<E_(min) is satisfied), the process proceeds tostep 150. If not (i.e., E_(u)≧E_(min) is satisfied), the processbypasses steps 150 and 160 so as to proceed to step 170. When decisionstep 140 is first executed, the process proceeds from step 140 to step150 because the least square error E_(min) is first set to asufficiently large value.

[0045] At step 150, the coefficients c₀, C₁, . . . , c₉ of the partialpolynomial are outputted to the extract unit 20. Then the value of theleast square error E_(min) is replaced with the value of the squareerror E_(u) at step 160. Thus the least square error E_(min) is updated.Further the partial polynomial generated at step 110 is stored, and anew variable x_(3+n) (n is a current repeat count) corresponding to theoutput z of the partial polynomial is added to the input variable groupat step 160. Then the process proceeds to step 170.

[0046] At step 170, it is determined whether the current repeat countreaches a predetermined number (e.g., 20 or 30), that is, whether acriterion for termination of the process is satisfied. If yes (i.e., itis determined at step 170 that the current repeat count reaches thepredetermined number), the process terminates. If not (i.e., it isdetermined at step 170 that the current repeat count does not reach thepredetermined number yet), the process returns to step 100 to repeatsteps 110-170. In this way, the pixel evaluation model corresponding tothe relationship f of formula (1) is generated.

[0047] More specifically, the pixel evaluation model is generated asfollows. Referring to FIG. 5, the variables x₁, x₂, x₃ corresponding tothe RGB values of the training image are selected at step 100, and thepartial polynomial A is generated using the variables x₁, x₂, x₃ at step110. That is, the coefficients c₀, c₁, . . . , c₉ of the partialpolynomial are determined at step 110. Then the square error E_(u) iscalculated at step 120. Assuming that it is determined at steps 130 and140 that E_(u)<E_(min) is satisfied, the coefficients c₀, c₁, . . . , c₉are outputted to the extract unit 20 at step 150 and the least squareerror E_(min) is updated using the square error E_(u) at step 160.Further the partial polynomial A is stored and a variable x₄corresponding to the output z of the partial polynomial A is added tothe input variable group at step 160.

[0048] Then the process proceeds to step 170 and returns to step 100.Three different variables (e.g., x₁, x₂, x₄) are selected from the inputvariable group (i.e., input variables x₁-x₄) at step 100, and a partialpolynomial B is generated using the selected variables x₁, x₂, x₄ atstep 110. Assuming that it is determined at steps 130 and 140 thatE_(u)<E_(min) is satisfied, the coefficients c₀, c₁, . . . , C₉ areoutputted to the extract unit 20 at step 150. Further the least squareerror E_(min) is updated, and the partial polynomial B is stored at step160. Thus the process is repeated so that a partial polynomial is newlygenerated and the new partial polynomial is employed as one of partialpolynomials that constitute the pixel evaluation model only when thesquare error E_(u) corresponding to the new partial polynomial is lessthan the current least square error E_(min) (i.e., the square errorE_(u) corresponding to the previously employed partial polynomial).

[0049] Assuming that two partial polynomials C, D are further employedas shown in FIG. 5 while execution of the process is repeated thepredetermined number of times, the pixel evaluation model generated as aresult of the repetitive execution of the process includes 4-layerpartial polynomials A, B, C, D. The output x₇ of the fourth-layerpartial polynomial D is provided as the output of the generated pixelevaluation model in this case.

[0050] The extract unit 20 receives the pixel evaluation model in theform of the coefficients c₀, c₁, . . . , c₉ of the layered partialpolynomials, which are outputted at step 150 of the model generationprocess. The extract unit 20 extracts the characteristic portion fromthe original image using the pixel evaluation model as follows.Referring to FIG. 6, at step 200, the feature value calculation block 22calculates the feature value for one pixel of the original image bysubstituting the RGB values of the pixel in the pixel evaluation model.

[0051] It is determined whether the feature value is equal to or largerthan 128 at step 210. If yes (i.e., it is determined at step 210 thatthe feature value is equal to or larger than 128), it is determined thatthe pixel belongs to the characteristic portion (i.e., face area) of theoriginal image. Therefore the process proceeds to step 220 to store thepixel as a pixel belonging to the characteristic portion. If it isdetermined at step 210 that the feature value is less than 128, theprocess bypasses step 220 so as to proceed to step 230.

[0052] At step 230, it is determined whether steps 200-220 have beenalready performed for all the pixels of the original image. If yes(i.e., it is determined at step 230 that steps 200-220 have been alreadyperformed for all the pixels), the process proceeds to step 240. If not(i.e., it is determined at step 230 that steps 200-220 have not beenperformed for all the pixels), steps 200-220 are repeated for the nextpixel. At step 240, the extract instruction block 23 defines, as anextractive area, a rectangular area so that the defined rectangular areaincludes all the pixels that have been stored at step 220. Then extractinstruction data that designates the rectangular area is generated atstep 250. The extract instruction data is binary data in which thevalues corresponding to the pixels belonging to the rectangular area are“1” and the values corresponding to the other pixels are “0”.

[0053] The image output block 24 extracts an image of the rectangulararea from the original image based on the extract instruction data, andoutputs the extracted image at step 260. Then the process terminates.Since the extract instruction data is provided in the form of binarydata, the image output block 24 or an external device which receives theextract instruction data can readily extract the designated portion.When the characteristic portion is thus extracted in the form of therectangular area, the outline of the face area is properly recognized inthe extracted image because the extracted image certainly includes theentire face area. In contrast, if an area that includes only pixelsbelonging to the face area is extracted, the extracted image does notnecessarily include the entire face area, that is, the extracted facearea may be notched.

[0054] The present image processing unit 1 may be implemented by aconventional computer system. However, it is preferable that themodeling unit 10 is implemented by a digital signal processor (DSP)dedicated to image processing so that the CPU which controls the extractunit 20 is not required to execute the processes of the modeling unit10, because the model generation block 13 should perform complexcalculation.

[0055] Further the blocks 11-13 of the modeling unit 10 and blocks 21-24of the extract unit 20 may be implemented by programs executable on acomputer. The programs may be stored in a computer readable medium suchas FD, MO, CD-ROM, DVD, or a hard disk. In this case, the programs areloaded from the computer readable medium into a memory of the computerwhen they should be executed. Alternatively, the programs may be storedin computer readable medium such as ROM or backup RAM. In this case, thecomputer readable medium is incorporated in the computer system thatimplements the present image processing unit 1.

[0056] The effects of the present embodiment are as follows. The extractunit 20 extracts the characteristic portion from the original imageusing the pixel evaluation model which the modeling unit 10 generatesusing the training image taken under the same condition (shootingcondition) under which the original image is taken. Therefore thepresent image processing unit 1 can adapt to change in the shootingcondition. That is, it can extract the characteristic portion properlyeven if the shooting condition changes.

[0057] Further the modeling unit 10 of the present image processing unit1 employs the modified GMDH as described above. The GMDH generatespartial polynomials corresponding to the respective pairs of inputvariables for each layer. In contrast, the modified GMDH generates onlyone partial polynomial for each layer. Thus the computational complexityis alleviated, and therefore the modeling unit 10 can generate the pixelevaluation model more rapidly.

[0058] Moreover, according to the modified GMDH, the generated partialpolynomial is employed only when the square error is reduced. It isobserved that a partial polynomial, which includes as an input variablethe output of a partial polynomial of the previous layer whose squareerror is relatively large (i.e., larger than the square error of themost superior partial polynomial of the previous layer), usually has arelatively large square error. Therefore the modified GMDH employs a newpartial polynomial only when the square error is reduced. As a result,the computational complexity is reduced without decreasing theprecision, and therefore the pixel evaluation model can be generatedmore efficiently.

[0059] Moreover in the present embodiment, when it is determined at step140 that a new partial polynomial is employed, the coefficients c₀, c₁,. . . , c₉ of the new partial polynomial is immediately outputted to theextract unit 20 at step 140. Therefore the present image processing unit1 can output the image of the characteristic portion more rapidly, ifsome decrease in the precision is allowable. Further the present imageprocessing unit 1 can output a less precise image of the characteristicportion as an interim output, and thereafter successively output moreprecise images.

[0060] Further, according to the modified GMDH, one partial polynomialthat includes three selected input variables is generated for eachlayer. Therefore, according to the modified GMDH, the pixel evaluationmodel which includes only one partial polynomial that includes threevariables corresponding to RGB values may be generated as an interimoutput as shown in FIG. 7A. If the extract unit 20 extracts thecharacteristic portion using such a pixel evaluation model, theextracted image can be outputted in real time, for example, at a rate of80 nano-seconds/pixel. In contrast, a pixel evaluation model generatedaccording to the GMDH includes at least two partial polynomials as shownin FIG. 7B, because partial polynomials that include two input variablesare generated for each layer according to the GMDH but the image data isin the form of three variables (i.e., the RGB values or YCrCb values).

[0061] The model generation process was executed according to themodified GMDH five times as trials. FIG. 8A shows the relationshipbetween the number of partial polynomials employed as a pixel evaluationmodel and the precision (i.e., the square error) of the pixel evaluationmodel, which were obtained as a result of the five trials. Further, themodel generation process was also executed according to the GMDH fivetimes as trials. FIG. 8B shows the relationship between the number ofpartial polynomials employed as a pixel evaluation model and theprecision (i.e., the square error) of the pixel evaluation model, whichwere obtained as a result of the five trials. The same image data thatincludes N=307200 pixels were used for all the trials.

[0062] Further FIG. 9 shows the result of the same trials in the form ofa table. The table includes the averages of the number of partialpolynomials that should be employed as the pixel evaluation model, andthe number of additions and multiplications that should be performedduring the image area extract process using the pixel evaluation modelfor achieving each of five discrete square errors.

[0063] It is found that the modified GMDH can achieve the same precisionby employing fewer partial polynomials as compared with the GMDH. Forexample, 10.2 partial polynomials are employed on average for achievingthe square error of 0.75×10⁹ or less according to the GMDH, while 5.2partial polynomials are employed on average for achieving the samesquare error according to the modified GMDH. The reason is that each ofthe partial polynomials generated according to the modified GMDH holds alot of information as compared with each of the partial polynomialsgenerated according to the GMDH.

[0064] Further FIG. 9 shows that the pixel evaluation model generatedaccording to the modified GMDH can achieve the same precision byperforming less computation (i.e., by performing fewer additions andmultiplications) as compared with the pixel evaluation model generatedaccording to the GMDH. In the case that the model generated according tothe modified GMDH is used for image area extraction, the computationthat should be performed for calculating one partial polynomial isslightly more as compared with the case that the model generatedaccording to the GMDH is used for image area extraction. However, thecomputation that should be performed for obtaining the output of theentire model is less because the number of partial polynomials includedin the model is reduced.

[0065] In this way, according to the present embodiment, the pixelevaluation model that includes the reduced number of partial polynomialsis generated rapidly, and consequently the responsiveness of the presentimage processing unit 1 is improved.

[0066] The supervisory image acquisition block 11 of the modeling unit10 corresponds to supervisory image acquisition means of the presentinvention. The training image acquisition block 12 corresponds totraining image acquisition means. The model generation block 13corresponds to model generation means. The original image acquisitionblock 21 of the extract unit 20 corresponds to the original imageacquisition means. The feature value calculation block 22 and theextract instruction block 23 together correspond to area definitionmeans. The 8-bit value of the i-th pixel of the supervisory imagecorresponds to a supervisory output provided by the i-th pixel. Themodel generation process shown in FIG. 4 corresponds to a processexecuted by the model generation means. The processes executed at steps200-250 of FIG. 6 correspond to processes executed by the areadefinition means.

Modifications

[0067] In the above embodiment, it may be determined at step 130 whetherthe square error meets another criterion. The criterion 20 is, forexample, that the square error is less than a predetermined referencevalue. In this case, the coefficients c₀, c₁, . . . , c₉ are outputtedat step 150 and further stored at step 160, only when the square erroris less than the predetermined reference value. If the criterion isappropriately determined, both of a certain 25 degree of responsivenessand a certain degree of precision can be ensured.

[0068] Alternatively, the coefficients c₀, c₁, . . . , c₉ may beoutputted at step 150 as the final output, when the square error meetsthe predetermined criterion. In this case, the criterion for terminationof the model generation process is also that the square error meets thepredetermined criterion. Therefore, after the coefficients are firstoutputted at step 150, it is determined at step 170 that the criterionfor termination is satisfied and therefore the model generation processterminates.

[0069] Further in the above embodiment, the training image acquisitionblock 12 and the original image acquisition block 21 may appropriatelyconvert (e.g., filter) the training image and the original imagerespectively so that a characteristic portion is properly identified.Thereby the extract unit 20 can properly extract a characteristicportion.

[0070] In the above embodiment, decision step 170 for determiningwhether the criterion for termination of model generation process issatisfied may be performed based on the number of the partialpolynomials that have been already employed, that is, the number of thevalidated partial polynomials.

[0071] In the above embodiment, the extract unit 20 may define, as anextractive area, an elliptical area that includes the face area insteadof the rectangular area. Alternatively, the extract unit 20 may define,as an extractive area, an area that includes only the pixels belongingto the characteristic portion.

[0072] In the above embodiment, all the pixels of the training image andthe supervisory image are not required to be used for calculating thesquare error E at steps 110 and 120, that is, for generating the pixelevaluation model. That is, some pixels which are sampled from thetraining image and the supervisory image may be used for calculating thesquare error E, because the characteristic portion of the image isrelatively large in area. For example, one pixel may be selected fromevery four consecutive pixels for calculating the square error E, andthe rest (i.e., the three pixels) may be discarded. In this case, themodeling unit 10 can generate the pixel evaluation model more rapidly,and therefore the responsiveness is further improved.

[0073] In the above embodiment, the training image and the originalimage are severally acquired using the respective cameras 40, 50.However, the image captured by the second camera 50 may be used as bothof the original image and the training image. In this case, the firstcamera 40 is not required.

[0074] In the above embodiment, each pixel of the supervisory image istwo-valued, that is, “0” or “255”. However, each pixel of thesupervisory image may be multi-valued.

[0075] In the above embodiment, the following partial polynomials may begenerated instead of the partial polynomial (2).

Z=c ₀ +c ₁ x _(p) +c ₂ x _(q) +c ₃ x _(r) +c ₄ x _(p) x _(q) +c ₅ x _(p)x _(r) +c ₆ x _(q) x _(r)  (4)

[0076] Further in the above embodiment, the modeling unit 10 may employthe GMDH as follows, because the number of the input variables of thepixel evaluation model to be generated (i.e., the number of variablesprovided as initial input variables when the pixel evaluation model isgenerated) is only three in this case. First, three polynomialscorresponding to the respective pair (x₁, x₂), (x₁, x₃), (x₂, x₃) of theinput variables x₁, x₂, x ₃ are generated as the first-layer partialpolynomials. Each of the polynomials are expressed as:

z _(k) =c ₀ +c ₁ x _(p) +c ₂ x _(q) +c ₃ x _(p) ² +c ₄ x _(q) ² +c ₅ x_(p) x _(q)  (5)

[0077] where c₀=c₅ are coefficients and (k, p, q) is (1, 1, 2) (2, 1, 3)or (3, 2, 3). Alternatively the following partial polynomials may begenerated instead of the partial polynomials (5).

z _(k) =c ₀ +c ₁ x _(p) +c ₂ x _(q) +c ₃ x _(p) x _(q)  .(6)

[0078] The values of the coefficients are determined using a linearregression analysis so that the square error E_(k) is minimized. Thesquare error E_(k) is expressed as:

E _(k)=Σ(y[i]−z _(k) [i]) ²  (7)

[0079] where Z_(k)[i] is the value of z when the RGB values of the i-thpixel of the training image is substituted in formula (6), y[i] is the 8bit value of the i-th pixel of the supervisory image, and the symbol “Σ”represents the summation for i. The variable i represents each of thepixels which are selected as training data from the training image orthe supervisory image.

[0080] When the values of the coefficients c₀-c₅ are determined, thesquare error Eku expressed as formula (7) is calculated using test data.In this case, the variable i represents each of the pixels which areselected as the test data from the training image or the supervisoryimage. If the least square error E¹ _(min) (i.e., the minimum of thesquare errors E_(k) ^(u)) is larger than the default least square errorE⁰ _(min) (i.e., E¹ _(min)>E⁰ _(min) is satisfied), the partialpolynomial corresponding to the least square error E¹ _(min) is selectedas the pixel evaluation model and the model generation processterminates. If E¹ _(min)>E⁰min is not satisfied, m₁ partialpolynomial(s) corresponding to smaller square error(s) E_(k) ^(u) areselected from the three partial polynomials z₁, z₂, z₃. The rest of thepartial polynomials z₁, z₂, z₃ are discarded.

[0081] Next m₁C₂ polynomials expressed as formula (6) are generated assecond-layer partial polynomials. Each of the generated partialpolynomials includes output variables of two first-layer polynomials asinput variables. The values of the coefficients c₀−C₅ are determinedusing training data, and then the square error E_(k) expressed asformula (7) is calculated using test data. If the least square error Emin of the second layer is larger than the least square error E¹ _(min)of the first layer (i.e., E² _(min)>E¹ _(min) is satisfied), thefirst-layer partial polynomial corresponding to the least square errorE¹min is selected as the pixel evaluation model and the model generationprocess terminates. If E² _(min)>E¹ _(min) is not satisfied, m₂ partialpolynomials corresponding to smaller square errors E_(k) are selectedfrom the m₁C₂ polynomials. The rest of the _(m1)C₂ polynomials arediscarded.

[0082] Thus the process is repeated for each layer (h-th layer). IfE^(h) _(min)>E^(h−1) _(min) is satisfied, the partial polynomialcorresponding to the least square error E^(h−1) _(min) of the previouslayer ((h−1)th layer) is selected as the pixel evaluation model. Furtherpartial polynomials of first-to (h−2)th layers involved in the selected(h−1)th-layer partial polynomial are selected as partial polynomialsincluded in the pixel evaluation model. Then the model generationprocess terminates. That is, the model generation process terminates ifthe least square error is increased.

[0083] In the above embodiment, the pixel evaluation model may beimplemented by a neural-network program, which consists of neurons andsynapses as shown in FIG. 10. However in this case, the responsivenessis lowered as compared with the above embodiment, because the neuralnetwork cannot be used as the complete pixel evaluation model until allthe weights corresponding to the respective synapses are determined.That is, the modeling unit 10 cannot output a less precise model as aninterim output.

[0084] In the above embodiment, the image processing unit 1 may extract,as a characteristic portion, an area other than the face area from theoriginal image. In this case, the training image should include thecharacteristic portion, and the white portion of the supervisory imageshould correspond to the characteristic portion of the training image.

[0085] The present invention is not limited to the above embodiment andmodifications, but may be variously embodied within the scope of theinvention.

What is claimed is:
 1. An image area extract unit for extracting acharacteristic portion from an original image, comprising: originalimage acquisition means for acquiring said original image; trainingimage acquisition means for acquiring a training image taken under asame condition under which said original image is taken; supervisoryimage acquisition means for acquiring a supervisory image thatdesignates a characteristic portion of said training image, saidsupervisory image including pixels each of which provides a supervisoryoutput, the characteristic portion of said training image correspondingto the characteristic portion of said original image; model generationmeans for generating a pixel evaluation model based on a relationshipbetween a value of a first pixel of said training image and thesupervisory output provided by a second pixel of said supervisory image,said second pixel corresponding to said first pixel, said pixelevaluation model receiving a value of one of pixels of said originalimage as an input and outputting a feature value of the pixel; and areadefinition means for calculating the feature value of each pixel of saidoriginal image by using said pixel evaluation model and defining anextractive area of said original image based on the feature value ofeach pixel of said original image, said extractive area including thecharacteristic portion of said original image.
 2. An image area extractunit as in claim 1, wherein said area definition means determineswhether each pixel of said original image belongs to the characteristicportion of said original image based on the feature value of the pixel,and defines said extractive area of said original image based on aresult of the determination.
 3. An image area extract unit as in claim1, wherein said model generation means sequentially generates aplurality of partial polynomials according to a Group Method of DataHandling so that an estimation model that includes at least one of saidplurality of partial polynomial is generated as said pixel evaluationmodel.
 4. An image area extract unit as in claim 3, wherein, if anevaluation value of a first partial polynomial generated as one of saidplurality of partial polynomials is improved as compared with anevaluation value of a second partial polynomial generated as one of saidplurality of partial polynomials previous to said first partialpolynomial, said model generation means employs said first partialpolynomial as the polynomial included in said pixel evaluation model andan output of said first partial polynomial is provided as an output ofsaid pixel evaluation model.
 5. An image area extract unit as in claim3, wherein, if an evaluation value of a partial polynomial generated asone of said plurality of partial polynomials satisfies a predeterminedcriterion, said model generation means employs said partial polynomialas the polynomial included in said pixel evaluation model and an outputof said partial polynomial is provided as an output of said pixelevaluation model.
 6. An image area extract unit as in claim 3, whereineach of said plurality of partial polynomials generated by said modelgeneration means includes three variables selected from an inputvariable group that includes a variable corresponding to a value of apixel of said training image
 7. An image area extract unit as in claim6, wherein each of said plurality of partial polynomials generated bysaid model generation means is expressed as: c ₀ +c ₁ x _(p) +c ₂ x _(q)+c ₃ x _(r) +c ₄ x _(p) ² +c ₅ x _(q) ² +c ₆ x _(r) ² +c ₇ x _(p) x _(q)+c ₈ x _(p) x _(r) +c ₉ x _(q) x _(r) where x_(p), x_(q), x_(r) are saidselected three variables and c₀, c₁, c₂, c₃, c₄, c₅, c₆, c₇, c₈, c₉ arecoefficients.
 8. An image area extract unit as in claim 6, wherein eachof said plurality of partial polynomials generated by said modelgeneration means is expressed as: c ₀ +c ₁ x _(p) +c ₂ x _(q) +c ₃ x_(r) +c ₄ x _(p) x _(q) +c ₅ x _(p) x _(r) +c ₆ x _(q) x _(r) wherex_(p), x_(q), x_(r) are said selected three variables and c₀, c₁, c₂,c₃, c₄, c₅, c₆ are coefficients.
 9. An image area extract unit as inclaim 1, wherein said original image acquisition means converts a firstimage taken under a condition into a second image in which acharacteristic portion of said first image is identified, and providessaid second image as said original image, and wherein said trainingimage acquisition means converts a third image, which is taken under thesame condition under which said first image is taken, into a fourthimage in which a characteristic portion of said third image isidentified, and provides said fourth image as said training image. 10.An image area extract unit as in claim 1, wherein said first pixel isone of representative pixels that are selected from all pixels of saidtraining image.
 11. An image area extract unit as in claim 1, whereinsaid area definition means generates, as a result of definition, extractinstruction data that designates pixels belonging to said extractivearea of said original image.
 12. A method for extracting acharacteristic portion from an original image, said method comprisingthe steps of: acquiring said original image; acquiring a training imagetaken under a same condition under which said original image is taken;acquiring a supervisory image that includes pixels corresponding topixels of said training image, said supervisory image designating acharacteristic portion of said training image, the characteristicportion of said training image corresponding to the characteristicportion of said original image; generating a pixel evaluation modelusing said training image and said supervisory image, said pixelevaluation model receiving a value of one of pixels of said originalimage and outputting a feature value of the pixel; calculating thefeature value of each pixel of said original image by using said pixelevaluation model; and defining an extractive area of said original imagebased on the feature value of each pixel of said original image, saidextractive area including the characteristic portion of said originalimage.
 13. A method as in claim 12, wherein said pixel evaluation modelincludes at least one partial polynomial.
 14. A method as in claim 13,wherein said generating step comprises the steps of: (a) selecting apredetermined number of variables from an input variable group thatincludes a variable corresponding to a value of a pixel of said trainingimage; (b) generating a first partial polynomial that includes saidselected variables as input variables based on a relationship between avalue of a first pixel of said training image and a value of a secondpixel of said supervisory image, said second pixel corresponding to saidfirst pixel; (c) calculating an evaluation value of said first partialpolynomial based on a relationship between a value of a third pixel ofsaid training image and a value of a fourth pixel of said supervisoryimage, said fourth pixel corresponding to said third pixel; (d) storingsaid first partial polynomial as the partial polynomial included in saidpixel evaluation model and adding an output variable of said firstpartial polynomial to said input variable group, if said evaluationvalue of said first partial polynomial satisfies a predeterminedcriterion for employment of a partial polynomial; and (e) repeating thesteps (a) to (d) if a predetermined criterion for termination of modelgeneration is not satisfied.
 15. A method as in claim 14, wherein thepredetermined number of variables that is selected at said selectingstep is
 3. 16. A method as in claim 15, wherein said first partialpolynomial generated at said generating step is expressed as: c ₀ +c ₁ x_(p) +c ₂ x _(q) +c ₃ x _(r) +c ₄ x _(p) ² +c ₅ x _(q) ² +c ₆ x _(r) ²+c ₇ x _(p) x _(q) +c ₈ x _(p) x _(r) +c ₉ x _(q) x _(r) where x_(p , x)_(q , x)are said selected three variables and c₀, c₁, c₂, c₃, c₄, c₅,c₆, c₇, c₈, c₉ are coefficients.
 17. A method as in claim 15, whereinsaid first partial polynomial generated at said generating step isexpressed as: c₀ +c ₁ x _(p) +c ₂ x _(q) +c ₃ x _(r) +c ₄ x _(p) x _(q)+c ₅ x _(p) x _(r) +c ₆ x _(q) x _(r) where x_(p), x_(q), x_(r) are saidselected three variables and c₀, c₁, c₂, c₃, c₄, c₅, c₆ arecoefficients.
 18. A method as in claim 14, wherein said predeterminedcriterion for employment of a partial polynomial is that said evaluationvalue of said first partial polynomial is improved as compared with anevaluation value of a second partial polynomial that is generated atsaid generating step previous to said first partial polynomial.
 19. Acomputer program for extracting a characteristic portion from anoriginal image, said computer program comprising: program code foracquiring said original image; program code for acquiring a trainingimage taken under a same condition under which said original image istaken; program code for acquiring a supervisory image that includespixels corresponding to pixels of said training image, said supervisoryimage designating a characteristic portion of said training image, thecharacteristic portion of said training image corresponding to thecharacteristic portion of said original image; program code forgenerating a pixel evaluation model based on a relationship between avalue of a first pixel of said training image and a value of a secondpixel of said supervisory image, said second pixel corresponding to saidfirst pixel, said pixel evaluation model receiving a value of one ofpixels of said original image and outputting a feature value of thepixel; program code for calculating the feature value of each pixel ofsaid original image by using said pixel evaluation model; and programcode for defining an extractive area of said original image based on thefeature value of each pixel of said original image, said extractive areaincluding the characteristic portion of said original image.
 20. Arecord medium which stores a computer program for extracting acharacteristic portion from an original image, said program comprising:program code for acquiring said original image; program code foracquiring a training image taken under a same condition under which saidoriginal image is taken; program code for acquiring a supervisory imagethat designates a characteristic portion of said training image, saidsupervisory image including pixels each of which provides a supervisoryoutput, the characteristic portion of said training image correspondingto the characteristic portion of said original image; program code forgenerating a pixel evaluation model based on a relationship between avalue of a first pixel of said training image and the supervisory outputprovided by a second pixel of said supervisory image, said second pixelcorresponding to said first pixel, said pixel evaluation model receivinga value of one of pixels of said original image and outputting a featurevalue of the pixel; program code for calculating the feature value ofeach pixel of said original image by using said pixel evaluation model;and program code for defining an extractive area of said original imagebased on the feature value of each pixel of said original image, saidextractive area including the characteristic portion of said originalimage.